---
title: "Supplementary appendix - Testing the link between regulatory agencies’ independence and credibility: Evidence from a survey experiment with regulatory stakeholders"
subtitle: "Version 1"
author: '**Saar Alon-Barkat**, **Madalina Busuioc** [alphabetical order]'
date: last-modified
format:
  html: 
    self-contained: true
    code-fold: true
    toc: true
    toc-location: left
execute:
  message: false
  warning: false
---




```{r silent-packages, echo = FALSE, eval = TRUE, message=FALSE, include = FALSE}
library(sjPlot)
library(tidyverse)
library(car)
library(reshape2)
library(viridis)
library(ggthemes)
library(corrr)
library(kableExtra)
library(dotwhisker)
library(sjmisc)
library(interflex)
library(effsize)
library(broom)
library(sjstats)
library(sjmisc)
library(stargazer)
library(AER)
library(ggpubr)
library(lavaan)


```

```{r set-global-options, echo = FALSE}
knitr::opts_chunk$set(eval = TRUE, 
                      echo = FALSE, 
                      message=FALSE,
                      warning = FALSE,
                      cache = FALSE,
                      fig.height = 4)

```


```{r , include=FALSE, echo=FALSE,warning=FALSE,message=FALSE}
#data



#load("C:/Users/SABarkat/Google Drive/University/R/EFSA/.RData")
source("~/R local/EFSA/code/efsa_dm_01.R")

efsa_00.t1 <- efsa_00 %>% 
  drop_na(independence.polit.efsa) %>% 
  data.frame() %>% 
  mutate(approve.pesticide = ban.pesticide %>% Recode("0=1;1=0"))

```

```{r, echo=FALSE, warning=FALSE, message=F}
#Figures and tables
table.x=0
figure.x=0
```

```{r}
set_theme(
  base=theme_tufte(),
  geom.outline.size = 0.01,
  geom.outline.color = "white", 
  geom.label.size = 3,
  geom.label.color = "grey50")

likert.scale = c("-5\nStrongly disagree",
                 "-4",
                 "-3",
                 "-2",
                 "-1",
                 "0",
                 "+1",
                 "+2",
                 "+3",
                 "+4",
                 "+5\nStrongly agree")
```

---

**Contents:**

1. Sample characteristics

2. Outcome variables

3. Summary statistics

4. Sample representativeness

5. Supplementary analyses

6. Survey


---

<br>

<br>

# Sample characteristics

<br>

In this section we provide additional details about the characteristics of our sample of participants.

<br>

## Sample

<br>


```{r, echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_00 %>% 
  drop_na(core.stakeholder) %>%  
  mutate(core.stakeholder.lab = core.stakeholder %>% Recode("0='Transparency register sample'; 1='Core stakeholders sample'"))

t2 <- t1 %>%
  group_by(core.stakeholder.lab) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```

<br>

*How did you receive the link for this survey?*

```{r, echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_raw %>% 
  filter(Q13.13!=0) %>% 
  mutate(Q13.13.lab = Q13.13 %>% Recode("1='1. I received an invitation from a person in my organization.';2='2. I received an invitation directly from the researchers.';3='3. Other'"))

t2 <- t1 %>%
  group_by(Q13.13.lab) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```


<br>

## Demographic characteristics

<br>


**Gender**
```{r, echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_00 %>% 
  drop_na(female) %>% 
  mutate(female.lab = Recode(female,"0='Men';1='Women'"))

t2 <- t1 %>%
  group_by(female.lab) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```

<br>

**Age**
```{r, echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_00 %>%
  drop_na(age) %>% 
  mutate(age.lab = Recode(age,"20='20s';
                                30='30s';
                                40='40s';
                                50='50s';
                                60='60s';
                                70='70s'")) 

t2 <- t1 %>%
  group_by(age.lab) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")
```

<br>

**Education level**

```{r}
t1 <- efsa_00 %>%
  drop_na(education) 

t2 <- t1 %>%
  group_by(education) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")
```

<br>

**Discipline**

```{r}
select(efsa_00,1) %>%
  mutate(education.discipline.1 = ifelse(efsa_raw$Q13.5_1==1,1,NA),
         education.discipline.2 = ifelse(efsa_raw$Q13.5_2==1,1,NA),
         education.discipline.3 = ifelse(efsa_raw$Q13.5_3==1,1,NA),
         education.discipline.4 = ifelse(efsa_raw$Q13.5_4==1,1,NA),
         education.discipline.5 = ifelse(efsa_raw$Q13.5_5==1,1,NA),
         education.discipline.6 = ifelse(efsa_raw$Q13.5_6==1,1,NA),
         education.discipline.7 = ifelse(efsa_raw$Q13.5_7==1,1,NA),
         education.discipline.8 = ifelse(efsa_raw$Q13.5_8==1,1,NA),
         education.discipline.9 = ifelse(efsa_raw$Q13.5_9==1,1,NA),
         education.discipline.10 = ifelse(efsa_raw$Q13.5_10==1,1,NA)) ->
    t1

t2 <- data.frame(discipline = c("Economics or Business Management",
                                   "Law",
                                   "Biology",
                                   "Chemistry",
                                   "Physics",
                                   "Engineering or Computer Science",
                                   "Political Science, Governance, Public Policy or Public Administration",
                                   "Communication",
                                   "Sociology",
                                   "Psychology")) %>% 
  mutate(n = c(filter(t1,education.discipline.1==1) %>% nrow(),
                        filter(t1,education.discipline.2==1) %>% nrow(),
                        filter(t1,education.discipline.3==1) %>% nrow(),
                        filter(t1,education.discipline.4==1) %>% nrow(),
                        filter(t1,education.discipline.5==1) %>% nrow(),
                        filter(t1,education.discipline.6==1) %>% nrow(),
                        filter(t1,education.discipline.7==1) %>% nrow(),
                        filter(t1,education.discipline.8==1) %>% nrow(),
                        filter(t1,education.discipline.9==1) %>% nrow(),
                        filter(t1,education.discipline.10==1) %>% nrow())) %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1))) %>% 
  arrange(desc(n))


t2 %>% 
  kable(col.names = c("Discipline",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```
<font size="0.5">

*Note: Participants could report more than one discipline.

</font>  

<br>

**Native language**

```{r}
select(efsa_00,1) %>%
  mutate(native.language.english = ifelse(efsa_raw$Q13.7==1,1,NA),
         native.language.French = ifelse(efsa_raw$Q13.8_1==1,1,NA),
         native.language.German = ifelse(efsa_raw$Q13.8_2==1,1,NA),
         native.language.Spanish = ifelse(efsa_raw$Q13.8_4==1,1,NA),
         native.language.Italian = ifelse(efsa_raw$Q13.8_5==1,1,NA),
         native.language.Dutch = ifelse(efsa_raw$Q13.8_6==1,1,NA),
         native.language.Portuguese = ifelse(efsa_raw$Q13.8_7==1,1,NA),
         native.language.other = ifelse(efsa_raw$Q13.8_8_TEXT!=0,1,NA)) ->
    t1

t2 <- data.frame(language = c("English",
                                   "French",
                                   "German",
                                   "Spanish",
                                   "Italian",
                                   "Dutch",
                                   "Portuguese",
                                   "Other")) %>% 
  mutate(n = c(filter(t1,native.language.english==1) %>% nrow(),
                        filter(t1,native.language.French==1) %>% nrow(),
                        filter(t1,native.language.German==1) %>% nrow(),
                        filter(t1,native.language.Spanish==1) %>% nrow(),
                        filter(t1,native.language.Italian==1) %>% nrow(),
                        filter(t1,native.language.Dutch==1) %>% nrow(),
                        filter(t1,native.language.Portuguese==1) %>% nrow(),
                        filter(t1,native.language.other==1) %>% nrow())) %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t3 <- t2 %>% 
  filter(language!="Other") %>% 
  arrange(desc(n)) %>% 
  rbind(t2 %>% filter(language=="Other"))


t3 %>% 
  kable(col.names = c("Language",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```
<font size="0.5">

*Note: Participants could report more than one native language.

</font> 

<br>


**Job task**
```{r}
t1 <- efsa_00 %>% 
  drop_na(job.task)

t2 <- data.frame(job.task = c("Research",
                                   "Human resource management",
                                   "Public communication or public relations",
                                   "Government relations, lobbying or advocacy",
                                   "Financing, marketing and sales",
                                   "Project management",
                                   "Administration")) %>% 
  mutate(n = c(filter(t1,job.task.research==1) %>% nrow(),
                        filter(t1,job.task.hr==1) %>% nrow(),
                        filter(t1,job.task.pr==1) %>% nrow(),
                        filter(t1,job.task.gr==1) %>% nrow(),
                        filter(t1,job.task.finance==1) %>% nrow(),
                        filter(t1,job.task.management==1) %>% nrow(),
                        filter(t1,job.task.administration==1) %>% nrow())) %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t3 <- t2 %>% 
  arrange(desc(n)) 


t3 %>% 
  kable(col.names = c("Job Task",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```
<font size="0.5">

*Note: Participants could report more than one job task.

</font> 



<br>



## Organizational characteristics

<br>


**Organization type**

```{r, echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_00 %>% 
  drop_na(organization.type.raw)

t2 <- t1 %>%
  group_by(organization.type.raw) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t3 <- t2 %>% 
  filter(organization.type.raw!="other") %>% 
  arrange(desc(n)) %>% 
  rbind(t2 %>% filter(organization.type.raw=="other"))

t3 %>% 
  kable(col.names = c("Organization type",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")

```


<br>

**Main policy domain of organization**

```{r}

select(efsa_00, 1) %>%
  mutate(
    organization.area.1 = ifelse(efsa_raw$Q1.5_1 == 1, 1, NA),
    organization.area.2 = ifelse(efsa_raw$Q1.5_2 == 1, 1, NA),
    organization.area.3 = ifelse(efsa_raw$Q1.5_3 == 1, 1, NA),
    organization.area.4 = ifelse(efsa_raw$Q1.5_4 == 1, 1, NA),
    organization.area.5 = ifelse(efsa_raw$Q1.5_5 == 1, 1, NA),
    organization.area.6 = ifelse(efsa_raw$Q1.5_6 == 1, 1, NA),
    organization.area.other = efsa_raw$Q1.5_7_TEXT
  ) ->
  t1


t2 <- data.frame(
  area = c(
    "Environment",
    "Telecommunications",
    "Food and Agriculture",
    "Health",
    "Financial services",
    "Energy",
    "Other"
  )
) %>%
  mutate(
    n = c(
      filter(t1, organization.area.1 == 1) %>% nrow(),
      filter(t1, organization.area.2 == 1) %>% nrow(),
      filter(t1, organization.area.3 == 1) %>% nrow(),
      filter(t1, organization.area.4 == 1) %>% nrow(),
      filter(t1, organization.area.5 == 1) %>% nrow(),
      filter(t1, organization.area.6 == 1) %>% nrow(),
      filter(t1, organization.area.other != 0) %>% nrow()
    )
  ) %>%
  mutate(percentage = ((n / nrow(t1) * 100) %>% round(1)))

t3 <- t2 %>%
  filter(area != "Other") %>%
  arrange(desc(n)) %>%
  rbind(t2 %>% filter(area == "Other"))


t3 %>%
  kable(col.names = c("Policy domain",
                      "n",
                      "%")) %>%
  kable_styling(
    bootstrap_options = c("condensed"),
    full_width = F,
    position = "left"
  )

```
<font size="0.5">

*Note: Participants could report more than one policy domain.

</font> 


<br>


## Participants' relations with EFSA 

<br>

I am well informed about the agency's mandate.


```{r}
efsa_00 %>% 
  drop_na(informed.efsa) %>% 
  ggplot(aes(x=affect.decisions.efsa)) +
  geom_bar(fill="dodgerblue2",width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5,labels = likert.scale)+
  theme_tufte()


```

<br>

The agency’s decisions may affect the operations of the organization in which I work.


```{r}
efsa_00 %>% 
  drop_na(affect.decisions.efsa) %>% 
  ggplot(aes(x=affect.decisions.efsa)) +
  geom_bar(fill="dodgerblue2",width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5,labels = likert.scale)+
  theme_tufte()


```

<br>

How often does your organization interact with the following agencies?

```{r}
plot_frq(efsa_00$interact.efsa,
        ylim = c(0,80),
        geom.colors = alpha("dodgerblue2",0.8),
        axis.title = "")+ 
  theme_tufte()
```


<br>

# Outcome variables

In this section we provide additional details about out outcome variables -- the three aspects of policy credibility. We report the distributions and correlations of the six survey items. Thereafter, we include the R code and results of our confirmatory factor analysis.   


## Distributions of items

Item 1: I believe that the opinion was based strictly on scientific considerations. 

```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q1, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
  scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
  facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")
```

<br>

Item 2: I believe that the opinion was influenced by political interests. (reversed)


```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q2, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
    scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")
```

<br>

Item 3: I believe that the opinion was affected by the industry interests. (reversed)


```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q3, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
    scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")

```

<br>

Item 4: I believe that the opinion was formed after evaluating all the relevant data and evidence. 

```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q4, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
    scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")
```

<br>

Item 5: I believe that the assessment of the evidence was based on adequate methods. 
```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q5, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
    scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")
```

<br>

Item 6: I believe that EFSA's opinion about the pesticide may change the next time it will be asked to reexamine it. (reversed)

```{r}
efsa_00 %>% 
  drop_na(ban.pesticide.lab) %>% 
  ggplot(aes(x=credibility.pest.q6, fill = ban.pesticide.lab)) +
  geom_bar(width = 0.75,alpha=0.8)+
  scale_x_continuous(name="",limits = c(-6,6),breaks = -5:5)+
    scale_fill_manual(values = c("dodgerblue2","dodgerblue4"))+
facet_wrap(~ban.pesticide.lab)+
  theme_tufte()+
  theme(legend.position = "none")
```

<br>

**Correlation matrix**

```{r, echo=FALSE,warning=FALSE,message=FALSE}
efsa_00 %>% 
  select(credibility.pest.q1:credibility.pest.q6) %>%
  
  tab_corr(triangle="lower",
           remove.spaces=T,
           p.numeric = T,
           fade.ns = F,
           digits = 2,
           na.deletion="pairwise",
           var.labels = str_c("Item ",1:6))
```


<br>


## Confirmatory factor analysis

We conducted our confirmatory factor analysis using `R lavaan` package. We first present a one-factor model, and then we  

<br>

**One-factor model:**
```{r,echo = TRUE}
cfa.model.1factor_01 <-
  'policy.credibility =~ credibility.pest.q1 +  credibility.pest.q2 + credibility.pest.q3 + credibility.pest.q4 + credibility.pest.q5 + credibility.pest.q6 '

fit.1factor_01 <- cfa(
  cfa.model.1factor_01,
  std.lv = TRUE,
  missing = "fiml",
  data = efsa_00.t1
)


summary(fit.1factor_01,fit.measures=T,standardize=T)

```

<br>

**Three-factors model (original)**

```{r,echo = TRUE}
cfa.model.3factor_01 <-
  'non.interference =~ credibility.pest.q1 +  credibility.pest.q2 + credibility.pest.q3
    expert.based  =~ credibility.pest.q4 + credibility.pest.q5 
    stability   =~ credibility.pest.q6 '

fit.3factor_01 <- cfa(
  cfa.model.3factor_01,
  std.lv = TRUE,
  missing = "fiml",
  data = efsa_00.t1
)

summary(fit.3factor_01,fit.measures=T,standardize=T)
```

<br>
**Three-factors model (modified)**

```{r,echo = TRUE}
cfa.model.3factor_02 <-
  'non.interference =~ credibility.pest.q2 + credibility.pest.q3
    expert.based  =~ credibility.pest.q1 +  credibility.pest.q4 + credibility.pest.q5 
    stability   =~ credibility.pest.q6 '

fit.3factor_02 <- cfa(
  cfa.model.3factor_02,
  std.lv = TRUE,
  missing = "fiml",
  data = efsa_00.t1
)

summary(fit.3factor_02,fit.measures=T,standardize=T)

```


```{r}
anova(fit.3factor_02,fit.3factor_01, fit.1factor_01)

```

<br>

**Factor Loadings (three-factors model)**

```{r}
parameterEstimates(fit.3factors, standardized=TRUE) %>% 
  filter(op == "=~") %>% 
  select('Latent Factor'=lhs, Indicator=rhs, B=est, SE=se, Z=z, 'p-value'=pvalue, Beta=std.all) %>% 
  mutate(Indicator = str_c("Item ",1:6)) %>% 
  kable(digits = 3)%>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")%>%
  collapse_rows(columns = 1, valign = "top")
```


<br>


# Summary statistics

```{r,results="asis", echo=FALSE,warning=FALSE,message=FALSE}
t1 <- efsa_00 %>% 
  select(credibility.pest.commitment,
         credibility.pest.expertise,
         credibility.pest.considerations,
         independence.polit.efsa,
         ban.pesticide,
         treatment,
         supports.pesticides,
         greediness.food.industry,
         trust.eu.commission,
         trust.eu.parliament,
         trust.eu.council,
         female,
         age,
         affect.decisions.efsa,
         informed.efsa,
         credibility.efsa.pre) %>% 
  descr() %>% 
  data.frame() %>% 
  select(-type,
         -label,
         -NA.prc,
         -se,
         -trimmed,
         -skew) %>% 
mutate_at(vars(c("mean","sd","md")),funs(round(.,3))) %>% 
  mutate(var = c(
"1. Credibility (stability)",
"2. Credibility (expert-based)",
"3. Credibility (non-interference)",
"4. Percevied political independence",
"5. Regulatory outcome (ban pesticide = 1)",
"6. Independence treatment",
"7. Supports pesticides",
"8. Perceived greediness of food industry",
"9. Trust in European Commission",
"10. Trust in European Parliament",
"11. Trust in Council of European Union",
"12. Gender (Female = 1)",
"13. Age",
"14. Organization affected by EFSA's decisions",
"15. Familiarity with EFSA's mandate",
"16. Perceived credibility of EFSA (pre manipulation)"
  ))
rownames(t1) <- 1:nrow(t1)



t1 %>% 
  kable(col.names = c("",
                      "Variable",
                      "n",
                      "Mean",
                      "SD",
                      "Median",
                      "Range")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")
```

<br>
```{r, echo=FALSE,warning=FALSE,message=FALSE}
efsa_00 %>% 
  select(credibility.pest.commitment,
         credibility.pest.considerations,
         credibility.pest.expertise,
         independence.polit.efsa,
         ban.pesticide,
         treatment,
         supports.pesticides,
         greediness.food.industry,
         trust.eu.commission,
         trust.eu.parliament,
         trust.eu.council,
         female,
         affect.decisions.efsa,
         informed.efsa,
         credibility.efsa.pre) %>%
  
  tab_corr(triangle="lower",
           remove.spaces=T,
           p.numeric = T,
           fade.ns = F,
           digits = 2,
           na.deletion="pairwise",
           var.labels = str_c("Variable ",1:15))
```

<br>

# Sample representativeness

<br>

To test the representativeness of our sample of participants, we collected demographic data about the population of EFSA’s stakeholders, using our raw lists of potential stakeholders which we obtained from both the core stakeholders group and the transparency register group. We randomly sampled 400 participants from these two group (200 from each), and asked a research assistant to search online for information on these individual stakeholders from publicly available websites (such as LinkedIn). We focused on two demographic criteria which were widely available: their gender and whether they have a PhD degree. The comparison between the survey sample and the random sample of 400 stakeholders from the raw list is presented in the table below. 

<br>

```{r}
coding.representation <- read_csv("C:/Users/SABarkat/Google Drive/University/R/EFSA/data/repersentation.csv")
```

```{r}
#across groups
coding.representation %>% 
  mutate(male = 1-female) %>% 
  pivot_longer(cols = c(phd,male)) %>% 
  group_by(power.group, name) %>% 
  summarise(n = n(),
            mean = mean(value,na.rm = T),
            sd = sd(value,na.rm = T)) %>% 
  mutate(se = sd/sqrt(n)) %>% 
  mutate(ci.low = mean - 1.96*se,
         ci.high = mean + 1.96*se) %>% 
  select(-sd,-se)

#combined
coding.representation %>% 
  mutate(male = 1-female) %>% 
  pivot_longer(cols = c(phd,male)) %>% 
  group_by(name) %>% 
  summarise(n = n(),
            mean = mean(value,na.rm = T),
            sd = sd(value,na.rm = T)) %>% 
  mutate(se = sd/sqrt(n)) %>% 
  mutate(ci.low = mean - 1.96*se,
         ci.high = mean + 1.96*se) %>% 
  select(-sd,-se)


```

<br>

```{r}
t1 <- efsa_00 %>%
  drop_na(female) %>% 
  filter(sample.group=="EU transparency register")

t2 <- t1 %>%
  group_by(female) %>% 
  tally() %>% 
  mutate(percentage = ((n/nrow(t1)*100) %>% round(1)))

t2 %>% 
  kable(col.names = c("",
                      "n",
                      "%")) %>% 
    kable_styling(bootstrap_options = c("condensed"),
                full_width = F,
                position = "left")
```


<br>


<br>

# Supplementary analyses

```{r}

tmod.considerations.null <- lm(credibility.pest.considerations ~
                                 1,
                               data = efsa_00.t1)


tmod.considerations.1 <- update(tmod.considerations.null, . ~ . +
                                  independence.polit.efsa)
tmod.considerations.2 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa
)


tmod.expertise.null <- update(tmod.considerations.null, credibility.pest.expertise ~ . )
tmod.expertise.1 <- update(tmod.considerations.1, credibility.pest.expertise ~ . )
tmod.expertise.2 <- update(tmod.considerations.2, credibility.pest.expertise ~ . )

tmod.commitment.null <- update(tmod.considerations.null, credibility.pest.commitment ~ . )
tmod.commitment.1 <- update(tmod.considerations.1, credibility.pest.commitment ~ . )
tmod.commitment.2 <- update(tmod.considerations.2, credibility.pest.commitment ~ . )


tmod1.1stage.1 <- lm(independence.polit.efsa~ 
                     treatment,
                data=efsa_00.t1)

tmod1.1stage.2 <- tmod1.1stage.1 %>% update(
  . ~ . +
    ban.pesticide+
    trust.eu.commission+
    trust.eu.parliament+
    trust.eu.council+
    affect.decisions.efsa+
    informed.efsa
)

tmod.rf.considerations.1 <- tmod1.1stage.1 %>% update(credibility.pest.considerations~.) 
  
tmod.rf.considerations.2 <- tmod1.1stage.2 %>% update(credibility.pest.considerations~.) 


tmod.rf.expertise.1 <- update(tmod.rf.considerations.1, credibility.pest.expertise ~ . ) 
tmod.rf.expertise.2 <- update(tmod.rf.considerations.2, credibility.pest.expertise ~ . ) 

tmod.rf.commitment.1 <- update(tmod.rf.considerations.1, credibility.pest.commitment ~ . ) 
tmod.rf.commitment.2 <- update(tmod.rf.considerations.2, credibility.pest.commitment ~ . ) 


tmod.considerations.11 <- update(tmod.considerations.2, . ~ . + 
                      supports.pesticides)
tmod.considerations.12 <- update(tmod.considerations.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.11 <- update(tmod.expertise.2, . ~ . + 
                      supports.pesticides)
tmod.expertise.12 <- update(tmod.expertise.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.11 <- update(tmod.commitment.2, . ~ . + 
                      supports.pesticides)
tmod.commitment.12 <- update(tmod.commitment.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

```



<br>

In the regression tables below, I report the results of additional robust analyses. 

*Tables A1--A3* replicate the regression tables in the paper while restricting our sample to those who answer correctly on the comprehension question (*n* = `r efsa_00.t1 %>% filter(comprehension.check == 1) %>% nrow()`). Next, in *Tables A4--A6* we exclude those who reported that they are not familiar with EFSA (*n* = `r efsa_00.t1 %>% filter(informed.efsa != -5) %>% nrow()`). Finally, in *Tables A7--A9*, we report the regression results when loading credibility item 1 ("I believe that the opinion was based strictly on scientific considerations") in the expert-based factor, instead of the non-interference. Finally, in Tables *A10–A11*, we test the robustness of our analyses for the adding of additional control variables.


<br>

## ***Excluding those who did not answer correctly on the comprehension question.***

<br>

**Regression table A1**

```{r}
efsa_00.robust.1 <- efsa_00.t1 %>% 
  filter(comprehension.check == 1)
```


(Table 2)

```{r}
tmod.considerations.null <- lm(credibility.pest.noninterference_q23 ~
                                 1,
                               data = efsa_00.robust.1)


tmod.considerations.1 <- update(tmod.considerations.null, . ~ . +
                                  independence.polit.efsa)
tmod.considerations.2 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa
)


tmod.expertise.null <- update(tmod.considerations.null, credibility.pest.expertise_q145 ~ . )
tmod.expertise.1 <- update(tmod.considerations.1, credibility.pest.expertise_q145 ~ . )
tmod.expertise.2 <- update(tmod.considerations.2, credibility.pest.expertise_q145 ~ . )

tmod.commitment.null <- update(tmod.considerations.null, credibility.pest.commitment_q6 ~ . )
tmod.commitment.1 <- update(tmod.considerations.1, credibility.pest.commitment_q6 ~ . )
tmod.commitment.2 <- update(tmod.considerations.2, credibility.pest.commitment_q6 ~ . )


tmod.credibility.null <- update(tmod.considerations.null, credibility.pest.index ~ . )
tmod.credibility.1 <- update(tmod.considerations.1, credibility.pest.index ~ . )
tmod.credibility.2 <- update(tmod.considerations.2, credibility.pest.index ~ . )

```

```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.1,
                  tmod.credibility.2,
                  tmod.considerations.1,
                  tmod.considerations.2,
                  tmod.expertise.1,
                  tmod.expertise.2,
                  tmod.commitment.1,
                  tmod.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence"),
                  order.terms = c(2,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2)
)
```

Table 3
```{r,echo=FALSE,warning=FALSE,message=FALSE}

tmod1.1stage.1 <- lm(independence.polit.efsa~ 
                     treatment,
                data=efsa_00.robust.1)

tmod1.2stage.1 <- tmod1.1stage.1 %>% update(.~.+
                     ban.pesticide+
                     trust.eu.commission+
                     trust.eu.parliament+
                     trust.eu.council+
                     affect.decisions.efsa+
                     informed.efsa)


tmod.rf.considerations.1 <- tmod1.1stage.1 %>% update(credibility.pest.noninterference_q23~.)
tmod.rf.considerations.2 <- tmod1.2stage.1 %>% update(credibility.pest.noninterference_q23~.)

tmod.rf.expertise.1 <- update(tmod1.1stage.1, credibility.pest.expertise_q145 ~ . ) 
tmod.rf.expertise.2 <- update(tmod1.2stage.1, credibility.pest.expertise_q145 ~ . ) 

tmod.rf.commitment.1 <- update(tmod1.1stage.1, credibility.pest.commitment_q6 ~ . ) 
tmod.rf.commitment.2 <- update(tmod1.2stage.1, credibility.pest.commitment_q6 ~ . ) 

tmod.rf.credibility.1 <- update(tmod1.1stage.1, credibility.pest.index ~ . ) 
tmod.rf.credibility.2 <- update(tmod1.2stage.1, credibility.pest.index ~ . ) 


```




```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.rf.credibility.1,tmod.rf.credibility.2,
                  tmod.rf.considerations.1,tmod.rf.considerations.2,
                  tmod.rf.expertise.1,tmod.rf.expertise.2,
                  tmod.rf.commitment.1,tmod.rf.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  string.est = " ",
                  digits = 3,
                  pred.labels = c("Constant","Independence treatment"),
                  order.terms = c(2,1),
                  string.p = "p-value",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
```



Table 4

```{r}
tmod.credibility.11 <- update(tmod.credibility.2, . ~ . + 
                      supports.pesticides)
tmod.credibility.12 <- update(tmod.credibility.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.considerations.11 <- update(tmod.considerations.2, . ~ . + 
                      supports.pesticides)
tmod.considerations.12 <- update(tmod.considerations.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.11 <- update(tmod.expertise.2, . ~ . + 
                      supports.pesticides)
tmod.expertise.12 <- update(tmod.expertise.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.11 <- update(tmod.commitment.2, . ~ . + 
                      supports.pesticides)
tmod.commitment.12 <- update(tmod.commitment.2, . ~ . + 
                      ban.pesticide*supports.pesticides)


```

```{r}
control.vars_2 <- c(
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.11,tmod.credibility.12,
                  tmod.considerations.11,tmod.considerations.12,
                  tmod.expertise.11,tmod.expertise.12,
                  tmod.commitment.11,tmod.commitment.12,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars_2,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence","Ban pesticide (0=approve)","Supports pesticides","Supports pesticides × Ban pesticide"),
                  order.terms = c(2:5,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
        

```


<br>


## ***Excluding those who are not familiar with EFSA.***

<br>

```{r}
efsa_00.robust.2 <- efsa_00.t1 %>% 
  filter(informed.efsa != -5)
```


(Table 2)

```{r}
tmod.considerations.null <- lm(credibility.pest.noninterference_q23 ~
                                 1,
                               data = efsa_00.robust.2)


tmod.considerations.1 <- update(tmod.considerations.null, . ~ . +
                                  independence.polit.efsa)
tmod.considerations.2 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa
)


tmod.expertise.null <- update(tmod.considerations.null, credibility.pest.expertise_q145 ~ . )
tmod.expertise.1 <- update(tmod.considerations.1, credibility.pest.expertise_q145 ~ . )
tmod.expertise.2 <- update(tmod.considerations.2, credibility.pest.expertise_q145 ~ . )

tmod.commitment.null <- update(tmod.considerations.null, credibility.pest.commitment_q6 ~ . )
tmod.commitment.1 <- update(tmod.considerations.1, credibility.pest.commitment_q6 ~ . )
tmod.commitment.2 <- update(tmod.considerations.2, credibility.pest.commitment_q6 ~ . )


tmod.credibility.null <- update(tmod.considerations.null, credibility.pest.index ~ . )
tmod.credibility.1 <- update(tmod.considerations.1, credibility.pest.index ~ . )
tmod.credibility.2 <- update(tmod.considerations.2, credibility.pest.index ~ . )

```

```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.1,
                  tmod.credibility.2,
                  tmod.considerations.1,
                  tmod.considerations.2,
                  tmod.expertise.1,
                  tmod.expertise.2,
                  tmod.commitment.1,
                  tmod.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence"),
                  order.terms = c(2,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2)
)
```

Table 3
```{r,echo=FALSE,warning=FALSE,message=FALSE}

tmod1.1stage.1 <- lm(independence.polit.efsa~ 
                     treatment,
                data=efsa_00.robust.2)

tmod1.2stage.1 <- tmod1.1stage.1 %>% update(.~.+
                     ban.pesticide+
                     trust.eu.commission+
                     trust.eu.parliament+
                     trust.eu.council+
                     affect.decisions.efsa+
                     informed.efsa)


tmod.rf.considerations.1 <- tmod1.1stage.1 %>% update(credibility.pest.noninterference_q23~.)
tmod.rf.considerations.2 <- tmod1.2stage.1 %>% update(credibility.pest.noninterference_q23~.)

tmod.rf.expertise.1 <- update(tmod1.1stage.1, credibility.pest.expertise_q145 ~ . ) 
tmod.rf.expertise.2 <- update(tmod1.2stage.1, credibility.pest.expertise_q145 ~ . ) 

tmod.rf.commitment.1 <- update(tmod1.1stage.1, credibility.pest.commitment_q6 ~ . ) 
tmod.rf.commitment.2 <- update(tmod1.2stage.1, credibility.pest.commitment_q6 ~ . ) 

tmod.rf.credibility.1 <- update(tmod1.1stage.1, credibility.pest.index ~ . ) 
tmod.rf.credibility.2 <- update(tmod1.2stage.1, credibility.pest.index ~ . ) 


```




```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.rf.credibility.1,tmod.rf.credibility.2,
                  tmod.rf.considerations.1,tmod.rf.considerations.2,
                  tmod.rf.expertise.1,tmod.rf.expertise.2,
                  tmod.rf.commitment.1,tmod.rf.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  string.est = " ",
                  digits = 3,
                  pred.labels = c("Constant","Independence treatment"),
                  order.terms = c(2,1),
                  string.p = "p-value",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
```



Table 4

```{r}
tmod.credibility.11 <- update(tmod.credibility.2, . ~ . + 
                      supports.pesticides)
tmod.credibility.12 <- update(tmod.credibility.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.considerations.11 <- update(tmod.considerations.2, . ~ . + 
                      supports.pesticides)
tmod.considerations.12 <- update(tmod.considerations.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.11 <- update(tmod.expertise.2, . ~ . + 
                      supports.pesticides)
tmod.expertise.12 <- update(tmod.expertise.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.11 <- update(tmod.commitment.2, . ~ . + 
                      supports.pesticides)
tmod.commitment.12 <- update(tmod.commitment.2, . ~ . + 
                      ban.pesticide*supports.pesticides)


```

```{r}
control.vars_2 <- c(
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.11,tmod.credibility.12,
                  tmod.considerations.11,tmod.considerations.12,
                  tmod.expertise.11,tmod.expertise.12,
                  tmod.commitment.11,tmod.commitment.12,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars_2,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence","Ban pesticide (0=approve)","Supports pesticides","Supports pesticides × Ban pesticide"),
                  order.terms = c(2:5,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
        

```

<br>


## ***Loading item CRED3 on non-interference***

Table 2
```{r}
efsa_00.robust.3 <- efsa_00.t1 %>%
  mutate(
    credibility.pest.noninterference_q123 = (credibility.pest.q1 +
                                             credibility.pest.q2 +
                                             credibility.pest.q3) / 3,
    credibility.pest.expertise_q45 = (credibility.pest.q4 +
                                             credibility.pest.q5) / 2
  )

```


(Table 2)

```{r}
tmod.considerations.null <- lm(credibility.pest.noninterference_q123 ~
                                 1,
                               data = efsa_00.robust.3)


tmod.considerations.1 <- update(tmod.considerations.null, . ~ . +
                                  independence.polit.efsa)
tmod.considerations.2 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa
)


tmod.expertise.null <- update(tmod.considerations.null, credibility.pest.expertise_q45 ~ . )
tmod.expertise.1 <- update(tmod.considerations.1, credibility.pest.expertise_q45 ~ . )
tmod.expertise.2 <- update(tmod.considerations.2, credibility.pest.expertise_q45 ~ . )

tmod.commitment.null <- update(tmod.considerations.null, credibility.pest.commitment_q6 ~ . )
tmod.commitment.1 <- update(tmod.considerations.1, credibility.pest.commitment_q6 ~ . )
tmod.commitment.2 <- update(tmod.considerations.2, credibility.pest.commitment_q6 ~ . )


tmod.credibility.null <- update(tmod.considerations.null, credibility.pest.index ~ . )
tmod.credibility.1 <- update(tmod.considerations.1, credibility.pest.index ~ . )
tmod.credibility.2 <- update(tmod.considerations.2, credibility.pest.index ~ . )

```

```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.1,
                  tmod.credibility.2,
                  tmod.considerations.1,
                  tmod.considerations.2,
                  tmod.expertise.1,
                  tmod.expertise.2,
                  tmod.commitment.1,
                  tmod.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence"),
                  order.terms = c(2,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2)
)
```

Table 3
```{r,echo=FALSE,warning=FALSE,message=FALSE}

tmod1.1stage.1 <- lm(independence.polit.efsa~ 
                     treatment,
                data=efsa_00.robust.3)

tmod1.2stage.1 <- tmod1.1stage.1 %>% update(.~.+
                     ban.pesticide+
                     trust.eu.commission+
                     trust.eu.parliament+
                     trust.eu.council+
                     affect.decisions.efsa+
                     informed.efsa)


tmod.rf.considerations.1 <- tmod1.1stage.1 %>% update(credibility.pest.noninterference_q123~.)
tmod.rf.considerations.2 <- tmod1.2stage.1 %>% update(credibility.pest.noninterference_q123~.)

tmod.rf.expertise.1 <- update(tmod1.1stage.1, credibility.pest.expertise_q45 ~ . ) 
tmod.rf.expertise.2 <- update(tmod1.2stage.1, credibility.pest.expertise_q45 ~ . ) 

tmod.rf.commitment.1 <- update(tmod1.1stage.1, credibility.pest.commitment_q6 ~ . ) 
tmod.rf.commitment.2 <- update(tmod1.2stage.1, credibility.pest.commitment_q6 ~ . ) 

tmod.rf.credibility.1 <- update(tmod1.1stage.1, credibility.pest.index ~ . ) 
tmod.rf.credibility.2 <- update(tmod1.2stage.1, credibility.pest.index ~ . ) 


```




```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.rf.credibility.1,tmod.rf.credibility.2,
                  tmod.rf.considerations.1,tmod.rf.considerations.2,
                  tmod.rf.expertise.1,tmod.rf.expertise.2,
                  tmod.rf.commitment.1,tmod.rf.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  string.est = " ",
                  digits = 3,
                  pred.labels = c("Constant","Independence treatment"),
                  order.terms = c(2,1),
                  string.p = "p-value",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
```



Table 4

```{r}
tmod.credibility.11 <- update(tmod.credibility.2, . ~ . + 
                      supports.pesticides)
tmod.credibility.12 <- update(tmod.credibility.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.considerations.11 <- update(tmod.considerations.2, . ~ . + 
                      supports.pesticides)
tmod.considerations.12 <- update(tmod.considerations.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.11 <- update(tmod.expertise.2, . ~ . + 
                      supports.pesticides)
tmod.expertise.12 <- update(tmod.expertise.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.11 <- update(tmod.commitment.2, . ~ . + 
                      supports.pesticides)
tmod.commitment.12 <- update(tmod.commitment.2, . ~ . + 
                      ban.pesticide*supports.pesticides)


```

```{r}
control.vars_2 <- c(
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.11,tmod.credibility.12,
                  tmod.considerations.11,tmod.considerations.12,
                  tmod.expertise.11,tmod.expertise.12,
                  tmod.commitment.11,tmod.commitment.12,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars_2,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence","Ban pesticide (0=approve)","Supports pesticides","Supports pesticides × Ban pesticide"),
                  order.terms = c(2:5,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
        

```


<br>

## ***Removing item CRED1***

Table 2
```{r}
efsa_00.robust.4 <- efsa_00.t1 %>%
  mutate(
    credibility.pest.index_exc_q2 = (credibility.pest.q1 +
                                      credibility.pest.q3 +
                                       credibility.pest.q4 +
                                       credibility.pest.q5 +
                                       credibility.pest.q6) / 5
  )

```


```{r}
tmod.considerations.null <- lm(credibility.pest.q3 ~
                                 1,
                               data = efsa_00.robust.4)


tmod.considerations.1 <- update(tmod.considerations.null, . ~ . +
                                  independence.polit.efsa)
tmod.considerations.2 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa
)


tmod.expertise.null <- update(tmod.considerations.null, credibility.pest.expertise_q145 ~ . )
tmod.expertise.1 <- update(tmod.considerations.1, credibility.pest.expertise_q145 ~ . )
tmod.expertise.2 <- update(tmod.considerations.2, credibility.pest.expertise_q145 ~ . )

tmod.commitment.null <- update(tmod.considerations.null, credibility.pest.commitment_q6 ~ . )
tmod.commitment.1 <- update(tmod.considerations.1, credibility.pest.commitment_q6 ~ . )
tmod.commitment.2 <- update(tmod.considerations.2, credibility.pest.commitment_q6 ~ . )


tmod.credibility.null <- update(tmod.considerations.null, credibility.pest.index_exc_q2 ~ . )
tmod.credibility.1 <- update(tmod.considerations.1, credibility.pest.index_exc_q2 ~ . )
tmod.credibility.2 <- update(tmod.considerations.2, credibility.pest.index_exc_q2 ~ . )

```

```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.1,
                  tmod.credibility.2,
                  tmod.considerations.1,
                  tmod.considerations.2,
                  tmod.expertise.1,
                  tmod.expertise.2,
                  tmod.commitment.1,
                  tmod.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence"),
                  order.terms = c(2,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2)
)
```

Table 3
```{r,echo=FALSE,warning=FALSE,message=FALSE}

tmod1.1stage.1 <- lm(independence.polit.efsa~ 
                     treatment,
                data=efsa_00.robust.4)

tmod1.2stage.1 <- tmod1.1stage.1 %>% update(.~.+
                     ban.pesticide+
                     trust.eu.commission+
                     trust.eu.parliament+
                     trust.eu.council+
                     affect.decisions.efsa+
                     informed.efsa)


tmod.rf.considerations.1 <- tmod1.1stage.1 %>% update(credibility.pest.q3~.)
tmod.rf.considerations.2 <- tmod1.2stage.1 %>% update(credibility.pest.q3~.)

tmod.rf.expertise.1 <- update(tmod1.1stage.1, credibility.pest.expertise_q145 ~ . ) 
tmod.rf.expertise.2 <- update(tmod1.2stage.1, credibility.pest.expertise_q145 ~ . ) 

tmod.rf.commitment.1 <- update(tmod1.1stage.1, credibility.pest.commitment_q6 ~ . ) 
tmod.rf.commitment.2 <- update(tmod1.2stage.1, credibility.pest.commitment_q6 ~ . ) 

tmod.rf.credibility.1 <- update(tmod1.1stage.1, credibility.pest.index_exc_q2 ~ . ) 
tmod.rf.credibility.2 <- update(tmod1.2stage.1, credibility.pest.index_exc_q2 ~ . ) 


```




```{r}
control.vars <- c(
  "ban.pesticide",
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "supports.pesticides",
  "organization.type.short"
)

sjPlot::tab_model(tmod.rf.credibility.1,tmod.rf.credibility.2,
                  tmod.rf.considerations.1,tmod.rf.considerations.2,
                  tmod.rf.expertise.1,tmod.rf.expertise.2,
                  tmod.rf.commitment.1,tmod.rf.commitment.2,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars,
                  string.est = " ",
                  digits = 3,
                  pred.labels = c("Constant","Independence treatment"),
                  order.terms = c(2,1),
                  string.p = "p-value",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
```



Table 4

```{r}
tmod.credibility.11 <- update(tmod.credibility.2, . ~ . + 
                      supports.pesticides)
tmod.credibility.12 <- update(tmod.credibility.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.considerations.11 <- update(tmod.considerations.2, . ~ . + 
                      supports.pesticides)
tmod.considerations.12 <- update(tmod.considerations.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.11 <- update(tmod.expertise.2, . ~ . + 
                      supports.pesticides)
tmod.expertise.12 <- update(tmod.expertise.2, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.11 <- update(tmod.commitment.2, . ~ . + 
                      supports.pesticides)
tmod.commitment.12 <- update(tmod.commitment.2, . ~ . + 
                      ban.pesticide*supports.pesticides)


```

```{r}
control.vars_2 <- c(
  "trust.eu.commission",
  "trust.eu.parliament",
  "trust.eu.council",
  "female",
  "affect.decisions.efsa",
  "informed.efsa",
  "organization.type.short"
)

sjPlot::tab_model(tmod.credibility.11,tmod.credibility.12,
                  tmod.considerations.11,tmod.considerations.12,
                  tmod.expertise.11,tmod.expertise.12,
                  tmod.commitment.11,tmod.commitment.12,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  rm.terms = control.vars_2,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  pred.labels = c("Constant","Perceived independence","Ban pesticide (0=approve)","Supports pesticides","Supports pesticides × Ban pesticide"),
                  order.terms = c(2:5,1),
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))
        

```



## ***Adding additional controls.***

<br>


**Regression table A10**

(table 2)
```{r, echo=FALSE,warning=FALSE,message=FALSE}


tmod.considerations.null <- lm(credibility.pest.noninterference_q23 ~
                                 1,
                               data = efsa_00.t1)


tmod.considerations.21 <- update(
  tmod.considerations.null,
  . ~ . +
    independence.polit.efsa +
    ban.pesticide +
    trust.eu.commission +
    trust.eu.parliament +
    trust.eu.council +
    affect.decisions.efsa +
    informed.efsa +
    female +
    age + 
    education.discipline.governance +
    education.discipline.economics +
    education.discipline.law + 
    education.discipline.chemistry +
    education.discipline.biology +
    native.language.english +
    native.language.French +
    native.language.German +
    native.language.Italian +
    native.language.Dutch +
    job.task.gr +
    job.task.management +
    job.task.research +
    job.task.pr +
    job.task.administration)


tmod.expertise.21 <- update(tmod.considerations.21, credibility.pest.expertise_q145 ~ . )

tmod.commitment.21 <- update(tmod.considerations.21, credibility.pest.commitment_q6 ~ . )

tmod.commitment.21 <- update(tmod.considerations.21, credibility.pest.commitment_q6 ~ . )

tmod.credibility.21 <- update(tmod.considerations.21, credibility.pest.index ~ . )

```


```{r}
sjPlot::tab_model(tmod.credibility.21,
                  tmod.considerations.21,
                  tmod.expertise.21,
                  tmod.commitment.21,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=1)
)
```



<br>


(table 4)

```{r, echo=FALSE,warning=FALSE,message=FALSE}

tmod.considerations.111 <- update(tmod.considerations.21, . ~ . + 
                      supports.pesticides)
tmod.considerations.121 <- update(tmod.considerations.21, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.expertise.111 <- update(tmod.expertise.21, . ~ . + 
                      supports.pesticides)
tmod.expertise.121 <- update(tmod.expertise.21, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.commitment.111 <- update(tmod.commitment.21, . ~ . + 
                      supports.pesticides)
tmod.commitment.121 <- update(tmod.commitment.21, . ~ . + 
                      ban.pesticide*supports.pesticides)

tmod.credibility.111 <- update(tmod.credibility.21, . ~ . + 
                      supports.pesticides)
tmod.credibility.121 <- update(tmod.credibility.21, . ~ . + 
                      ban.pesticide*supports.pesticides)

```


<br>

```{r}

sjPlot::tab_model(tmod.credibility.111,
          tmod.credibility.121,
          tmod.considerations.111,
          tmod.considerations.121,
          tmod.expertise.111,
          tmod.expertise.121,
          tmod.commitment.111,
          tmod.commitment.121,
                  show.ci = F,
                  show.se = T,
                  collapse.se = T,
                  emph.p = F,
                  digits = 3,
                  string.p = "p-value",
                  string.est = " ",
                  dv.labels = rep(c("Credibility (all)","Non-interference","Expert-based","Stability"),each=2))

```



<br>

# Survey

<br>

Below is the full text of the survey. Additional comments are presented in square brackets. The original Qualtrics file is available upon request. 

<br>

**Expert Survey - Regulatory Policy in the EU**  
   

Dear Participant, 

We are academic researchers from the Institute of Public Administration, Leiden University, studying regulatory policy in the European Union. We are interested in exploring how relevant stakeholders across the EU evaluate specific aspects of regulatory policy.     

As a stakeholder of some of the EU regulatory bodies, we believe that you have specific knowledge and expertise that is invaluable to our study. Therefore, we would be grateful if you would be willing to contribute to our study by filling in this **expert survey**.      

The survey begins with general questions about your evaluation of EU regulatory institutions, as well as challenges faced by these bodies. Thereafter, you will be asked about a specific regulatory policy area, in your case - food safety policy. **Filling in the survey takes approximately 10-15 minutes**.       

The survey is voluntary and **anonymous**. You will not be asked to identify yourself or your organization. The data collected will be used for research purposes only. You may withdraw from the study at any time. Please fill in the survey individually.      

For any further questions about the research, you may contact us personally.         
 
 
Sincerely, 

**AUTHORS' NAMES**   
**UNIVERSITY DEPARTMENT**


<br>

In order to indicate your consent to participate in the research, please press below: 
o	I agree to participate in the study. 

<br>

---


Before filling the expert survey, please answer the questions below:

<br>

Which of the following regulatory policy areas are most relevant to the operations of your organization? 

1. Environment  

2. Telecommunications  

3. Food and Agriculture   

4.	Health  

5. Financial services   

6.	Energy  

7.	Other:  ________________________________________________


<br>


In which type of organization do you currently work? 

1. Consultancy firm 

2.	Company   

3.	Trade or business association 

4.	Professional association  

5.	Non-governmental nonprofit organization 

6.	Research institute

7.	Law firm 

8.	Other:  ________________________________________________


<br>

---



First, we would like to ask you a few general questions regarding your opinion about several EU institutions and bodies. 

<br>

To what extent do you trust the following EU institutions?

                                       Don't   
                                       trust                                                  Highly 
                                      at all                       Neutral                     trust
                                        -5    -4    -3    -2    -1    0   +1    +2    +3   +4   +5 
    European Commission
    European Parliament 
    Council of the European Union

<br>
[Independence treatment condition]

Below is a list of selected EU agencies that operate independently of the EU political institutions.  
To what extent do you perceive these agencies to be credible regulators?

                                       Don't   
                                       trust                                                  Highly 
                                      at all                       Neutral                     trust
                                        -5    -4    -3    -2    -1    0   +1    +2    +3   +4   +5 
    European Environment Agency (EEA)
    European Food Safety Authority (EFSA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)


<br>

[Independence control condition]

Below is a list of selected EU agencies. 
To what extent do you perceive these agencies to be credible regulators?

                                       Don't   
                                       trust                                                  Highly 
                                      at all                       Neutral                     trust
                                        -5    -4    -3    -2    -1    0   +1    +2    +3   +4   +5 
    European Environment Agency (EEA)
    European Food Safety Authority (EFSA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)

<br>


Now we would like to ask you a few general questions regarding your opinion about private companies and corporations in the EU. 

<br>

Some people argue that most private companies and corporations do not care about the social consequences of their actions and products. They are only interested in maximizing their short-term profits.

To what extent do you agree or disagree with this view?

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

<br> 

We now wish to repeat the last question, with regard to private companies and corporations within specific industries.
To what extent do you agree or disagree with the following views?


In the pharmaceutical industry, most private companies and corporations do not care about the social consequences of their actions and products. They are only interested in maximizing their short-term profits.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

In the food industry, most private companies and corporations do not care about the social consequences of their actions and products. They are only interested in maximizing their short-term profits.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    
<br>

Now, we want to ask your opinion about contemporary challenges faced by EU regulatory bodies. 

<br>

New technologies often spark debate about the opportunities they bring versus the threats they pose. This also applies in the EU policy context.

For each of the following technologies, please indicate whether you believe that their implementation mainly brings benefits and opportunities, or rather poses a threat:

                              Mainly                                                  Mainly
                              poses                                                   brings
                              a threat                   Neutral                     benefits
                               -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    Artificial intelligence
    Genetically modified foods
    Blockchain
    Use of pesticides
    3D printing 
    Artificial food additives

<br>

In recent years, governments have been assisted by Machine Learning algorithms in decision making in a number of policy domains, including healthcare, education and the criminal system.   

In your opinion, would the use of algorithms by public agencies in the EU mainly improve or reduce the quality of their policy decisions?

    Mainly                                                    Mainly         I don't
    reduce                                                    improve          know
    quality                                                   quality
      -5    -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

In your opinion, would the use of algorithms by public agencies in the EU mainly improve or reduce the fairness of their decision-making process?

    Mainly                                              Mainly            I don't
    reduce                                              improve           know
    fairness                                            fairness
    -5    -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

<br>

Does your organization use machine learning algorithms to make decisions?

1. No

2. Yes

3. I am not sure

4. I prefer not to answer


<br>


---

In the following section, we present you with a recent case of a regulatory risk assessment on food safety policy. After presenting the case, we will ask you several questions regarding your opinion about it.   

<br>

**Risk assessment of pesticides**

<br>

In the European Union, the marketing and use of pesticides and other plant protection products is regulated by a large body of EU legislation. The European Commission regulates the risks of pesticide use on public health and the environment. The Commission’s decision whether to approve or restrict a pesticide’s active substance is based on a risk assessment conducted by **EFSA - the European Food Safety Authority**.      

<br>

[Independence treatment]

**EFSA** is a EU regulatory agency that operates **independently** of the European Commission, the European Parliament, and the Member States.      

The agency was legally established in 2002 under the General Food Law - Regulation (EC) 178/2002. According to its founding regulation, EFSA is “to serve as a point of reference by virtue of its *independence*” and the members of its scientific committee and panels “shall undertake to act *independently of any external influence*”. The regulation establishes the agency's legal, financial, and regulatory independence. EFSA's scientific opinions are based solely on judgments of independent scientists, *without any intervention* by the agency’s board or any other political bodies.       

Moreover, EFSA explicitly acknowledges the independence of its experts, methods and data from any external influence as one of its key organizational values, and has developed specific policies to ensure their **impartiality and neutrality**.


[Independence control]

EFSA is a EU regulatory agency that was set up to be a source of **advice and communication** on risks associated with the food chain. 

The agency was legally established in 2002 under the General Food Law - Regulation (EC) 178/2002. It is responsible for risk assessment and also has a duty to communicate its scientific findings to the public. According to its founding regulation, EFSA is to “provide scientific *advice* and... support for the Community's legislation and policies” and is expected to “act in close collaboration with the Commission and the Member States to promote the necessary coherence in the risk *communication* process.” The regulation specifies EFSA´s legal mandate, as well as its organizational, financial, and regulatory arrangements. 

EFSA acknowledges the importance of the effective **communication of its opinions** to decision‐makers and the general public, and has developed specific policies for this purpose

<br>

[Approve pesticide]

In 2015, EFSA submitted its conclusion on the risk assessment of the pesticide **"Flupyradifurone"**, a new active substance, which has been developed for protecting various crops against insects such as aphids, hoppers and whiteflies. EFSA was asked by the European Commission to evaluate the safety of this pesticide before making a decision about its approval. 
 
On the basis of the available data, EFSA concluded in its report that **Flupyradifurone** is not carcinogenic nor toxic for reproduction. It also noted that the conditions for the consideration of endocrine disrupting properties for human health (i.e. interferes with the hormone system) were not met. However, EFSA also noted that the information available was insufficient to entirely rule out the potential for endocrine disrupting effects of the pesticide. Also, it could not completely rule out the risk for consumers due to the unknown nature of residues that might be present in drinking water. In the end, EFSA did not propose to classify the pesticide as toxic, and concluded that it may be expected to fulfil the safety requirements laid down in the legislation. 
 
**Based on EFSA's conclusions, the pesticide Flupyradifurone has been approved**.     

<br>

[Approve pesticide]

In 2015, EFSA submitted its conclusion on the risk assessment of the pesticide **"Amitrole"**, which has been widely used as a weedkiller in orchards, grapes, olives and for non-crop uses. EFSA was asked by the European Commission to evaluate the safety of this pesticide before making a decision about the renewal of its approval. 
 
On the basis of the available data, EFSA identified critical areas of concern regarding the use of Amitrole. A high risk to operators, workers and bystanders from the use of **Amitrole** was identified. Additionally, EFSA concluded in its review that there is a high potential for groundwater exposure by toxic substances above the acceptable limit, as well as noting that Amitrole should be considered to have endocrine disrupting properties  (i.e. interferes with the hormone system). Finally, in its conclusion, it indicated that during the peer review it was proposed that Amitrole should be re-classified to a higher hazard category as “presumed human reproductive toxicant”. Substances are classified in this category when data provides clear evidence of an adverse effect for instance, on sexual function and fertility.  
   
**Based on EFSA's conclusions about these identified high risks, the pesticide Amitrole has been banned**.      

<br>

---

The following sentences refer to the abovementioned opinion of EFSA about the risk of the pesticide. Please state your agreement with each sentence.

<br>

[Questions were presented in a random order]


I believe that the opinion was based strictly on <u>scientific considerations</u>.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

I believe that the opinion was influenced by <u>political interests</u>.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

I believe that the opinion was influenced by <u>industry interests</u>.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

I believe that the opinion was formed after <u>evaluating all relevant data and evidence</u>.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

I believe that the assessment of the evidence was <u>based on adequate methods</u>. 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

I believe that EFSA's opinion about the pesticide may change the next time it is asked to re-examine it.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

<br>

Now we would like to ask you more general questions about EFSA. 
Please state your agreement with each of the following sentences. 

<br>

[Questions were presented in a random order]

EFSA’s opinions can be <u>influenced by EU political institutions</u> (European Commission, Council, Parliament).

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA’s opinions can be <u>influenced by the industry</u>.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA is a <u>credible</u> regulator.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA follows a <u>stable and consistent</u> approach in its risk assessment.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA's employees are highly skilled in their profession. 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA's employees understand the problems and issues in the field. 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA cooperates well with experts in the field.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA sets new scientific standards.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA’s output is of high quality.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA is an effective organization.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA is a competent regulator.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA’s mission is ethically defensible (their mission is the right mission). 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA has a positive influence on society. 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    
Decision-making in EFSA follows due process. 

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 

EFSA follows correct procedures.

    Strongly                                                 Strongly
    disagree                                                   agree
       -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 


<br>

Please complete the following sentence:

EFSA has a reputation for...

________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________



---


Now, we would like to ask you some general questions on the relationship between your organization and EFSA, as well as the other three EU agencies mentioned at the beginning of the survey (EEA, EMA, ECHA). 

Please state your agreement with the following sentences, for each agency. 

I am well informed about the agency's mandate.

                                   Strongly                                                 Strongly
                                   disagree                                                   agree
                                      -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    European Environment Agency (EEA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)
    European Food Safety Authority (EFSA)

The agency’s decisions may affect the operations of the organization in which I work.

                                   Strongly                                                 Strongly
                                   disagree                                                   agree
                                      -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    European Environment Agency (EEA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)
    European Food Safety Authority (EFSA)

How often does your organization interact with the following agencies?

                                          Not     About   About   About   A few     I    I prefer
                                          at      once    twice    once    times   don't  not to
                                          all     a year  a year  a month a month  know   answer
                                           1        2       3       4       5       6       7
    European Environment Agency (EEA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)
    European Food Safety Authority (EFSA)

<br>

Please state your agreement with the following sentences, for each agency:


The agency's opinions can be <u>influenced by EU political institutions</u> (European Commission, Council, Parliament).

                                   Strongly                                                 Strongly
                                   disagree                                                   agree
                                      -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    European Environment Agency (EEA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)

The agency's opinions can be <u>influenced by the industry</u>.

                                   Strongly                                                 Strongly
                                   disagree                                                   agree
                                      -5   -4    -3    -2    -1    0   +1    +2    +3    +4    +5 
    European Environment Agency (EEA)
    European Medicines Agency (EMA)
    European Chemicals Agency (ECHA)


---


Before finishing, we would like to ask you a few general questions about yourself:


What is your age?
    
1. Under 30

2. 30-40

3. 41-50

4. 51-60

5. 61-70

6. Older than 70

<br>

What is your gender?

1. Man

2. Woman

3. I prefer not to answer

<br>



What is your highest level of education?

1. High school graduate

2. Bachelor's degree

3. Master's degree

4. Doctorate degree

<br>


What is the disciplinary field of your highest academic degree? 

1. Economics or Business Management  

2. Law

3. Biology  

4. Chemistry

5. Physics

6. Engineering or Computer Science  

7. Political Science, Governance, Public Policy or Public Administration  

8. Communication

9. Sociology

10. Psychology

11. Other: ________________________________________________


<br>


In which of the following aspects are you mostly involved in your current position in your organization (you may select multiple options)

1. Research  

2. Human resource management

3. Public communication or public relations

4. Government relations, lobbying or advocacy

5. Financing, marketing and sales

6. Project management

7. Administration

8. Other: ________________________________________________


<br>



Are you a native English speaker?

1. Yes

2. No


<br>


What is your native language?

1. French

2. German

3. Spanish

4. Italian

5. Dutch

6. Portuguese

7. Other: ________________________________________________

<br>

Finally, we would like to ask you an informative question about the case that we presented to you (EFSA's risk assessment of the pesticide [**Flupyradifurone**/**Amitrole**]).

Please select the correct statement. 

1. EFSA identified high risks for the use of the pesticide [Flupyradifurone/Amitrole], and the pesticide was <u>banned</u>.

2. EFSA did not identify high risks for the use of pesticide [Flupyradifurone/Amitrole], and the pesticide was <u>approved</u>.  


<br>

Were you familiar with EFSA's opinion about the pesticide ["Flupyradifurone"/"Amitrole"] prior to this survey?

1. Yes

2. No


<br>

How did you receive the link for this survey?

1. I received an invitation from a person in my organization.

2. I received an invitation directly from the researchers.  

3. Other: ________________________________________________

<br>

Did you invite additional employees in your organization to participate in the study?

1. Yes

2. No

<br>

Is there anything else that you would like to share with us?

________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

<br>

Thank you very much for participating in the study.      

You may also invite other executives and staff members in your organization to fill this survey, by sharing the following link:

In case you would like to receive a summary of the findings of this study, please reply to our email below and mention this.

You may also contact us for any query about the research.


Sincerely,      

AUTHORS
UNIVERSITY DEPARTMENT


